{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Processors and Utils\n",
    "\n",
    "Description of the main tools and utilities that one needs to prepare the data for a `WideDeep` model constructor. \n",
    "\n",
    "#### The `preprocessing`  module\n",
    "\n",
    "There are 4 preprocessors, corresponding to 4 main components of the `WideDeep` model. These are\n",
    "\n",
    "* `WidePreprocessor`\n",
    "* `TabPreprocessor`\n",
    "* `TextPreprocessor`\n",
    "* `ImagePreprocessor` \n",
    "\n",
    "Behind the scenes, these preprocessors use a series of helper funcions and classes that are in the `utils` module. If you were interested please go and have a look to the documentation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  1. WidePreprocessor\n",
    "\n",
    "The `wide` component of the model is a linear model that in principle, could be implemented as a linear layer receiving the result of on one-hot encoding categorical columns. However, this is not memory efficient. Therefore, we implement a liner layer as an Embedding layer plus a bias. I will explain in a bit more detail later. \n",
    "\n",
    "With that in mind, `WidePreprocessor` simply encodes the categories numerically so that they are the indexes of the lookup table that is an Embedding layer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/javierrodriguezzaurin/.pyenv/versions/3.10.13/envs/widedeep310/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import pytorch_widedeep as wd\n",
    "\n",
    "from pytorch_widedeep.datasets import load_adult\n",
    "from pytorch_widedeep.preprocessing import WidePreprocessor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>fnlwgt</th>\n",
       "      <th>education</th>\n",
       "      <th>educational-num</th>\n",
       "      <th>marital-status</th>\n",
       "      <th>occupation</th>\n",
       "      <th>relationship</th>\n",
       "      <th>race</th>\n",
       "      <th>gender</th>\n",
       "      <th>capital-gain</th>\n",
       "      <th>capital-loss</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>native-country</th>\n",
       "      <th>income</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>25</td>\n",
       "      <td>Private</td>\n",
       "      <td>226802</td>\n",
       "      <td>11th</td>\n",
       "      <td>7</td>\n",
       "      <td>Never-married</td>\n",
       "      <td>Machine-op-inspct</td>\n",
       "      <td>Own-child</td>\n",
       "      <td>Black</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>38</td>\n",
       "      <td>Private</td>\n",
       "      <td>89814</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>9</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Farming-fishing</td>\n",
       "      <td>Husband</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>50</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>28</td>\n",
       "      <td>Local-gov</td>\n",
       "      <td>336951</td>\n",
       "      <td>Assoc-acdm</td>\n",
       "      <td>12</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Protective-serv</td>\n",
       "      <td>Husband</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&gt;50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>44</td>\n",
       "      <td>Private</td>\n",
       "      <td>160323</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>10</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Machine-op-inspct</td>\n",
       "      <td>Husband</td>\n",
       "      <td>Black</td>\n",
       "      <td>Male</td>\n",
       "      <td>7688</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&gt;50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>18</td>\n",
       "      <td>?</td>\n",
       "      <td>103497</td>\n",
       "      <td>Some-college</td>\n",
       "      <td>10</td>\n",
       "      <td>Never-married</td>\n",
       "      <td>?</td>\n",
       "      <td>Own-child</td>\n",
       "      <td>White</td>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>30</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age  workclass  fnlwgt     education  educational-num      marital-status  \\\n",
       "0   25    Private  226802          11th                7       Never-married   \n",
       "1   38    Private   89814       HS-grad                9  Married-civ-spouse   \n",
       "2   28  Local-gov  336951    Assoc-acdm               12  Married-civ-spouse   \n",
       "3   44    Private  160323  Some-college               10  Married-civ-spouse   \n",
       "4   18          ?  103497  Some-college               10       Never-married   \n",
       "\n",
       "          occupation relationship   race  gender  capital-gain  capital-loss  \\\n",
       "0  Machine-op-inspct    Own-child  Black    Male             0             0   \n",
       "1    Farming-fishing      Husband  White    Male             0             0   \n",
       "2    Protective-serv      Husband  White    Male             0             0   \n",
       "3  Machine-op-inspct      Husband  Black    Male          7688             0   \n",
       "4                  ?    Own-child  White  Female             0             0   \n",
       "\n",
       "   hours-per-week native-country income  \n",
       "0              40  United-States  <=50K  \n",
       "1              50  United-States  <=50K  \n",
       "2              40  United-States   >50K  \n",
       "3              40  United-States   >50K  \n",
       "4              30  United-States  <=50K  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = load_adult(as_frame=True)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "wide_cols = [\n",
    "    \"education\",\n",
    "    \"relationship\",\n",
    "    \"workclass\",\n",
    "    \"occupation\",\n",
    "    \"native-country\",\n",
    "    \"gender\",\n",
    "]\n",
    "crossed_cols = [(\"education\", \"occupation\"), (\"native-country\", \"occupation\")]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "wide_preprocessor = WidePreprocessor(wide_cols=wide_cols, crossed_cols=crossed_cols)\n",
    "X_wide = wide_preprocessor.fit_transform(df)\n",
    "# From here on, any new observation can be prepared by simply running `.transform`\n",
    "# new_X_wide = wide_preprocessor.transform(new_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  1,  17,  23, ...,  89,  91, 316],\n",
       "       [  2,  18,  23, ...,  89,  92, 317],\n",
       "       [  3,  18,  24, ...,  89,  93, 318],\n",
       "       ...,\n",
       "       [  2,  20,  23, ...,  90, 103, 323],\n",
       "       [  2,  17,  23, ...,  89, 103, 323],\n",
       "       [  2,  21,  29, ...,  90, 115, 324]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_wide"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the label encoding starts from `1`. This is because it is convenient to leave `0` for padding, i.e. unknown categories. Let's take from example the first entry"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  1,  17,  23,  32,  47,  89,  91, 316])"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_wide[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>education</th>\n",
       "      <th>relationship</th>\n",
       "      <th>workclass</th>\n",
       "      <th>occupation</th>\n",
       "      <th>native-country</th>\n",
       "      <th>gender</th>\n",
       "      <th>education_occupation</th>\n",
       "      <th>native-country_occupation</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>11th</td>\n",
       "      <td>Own-child</td>\n",
       "      <td>Private</td>\n",
       "      <td>Machine-op-inspct</td>\n",
       "      <td>United-States</td>\n",
       "      <td>Male</td>\n",
       "      <td>11th-Machine-op-inspct</td>\n",
       "      <td>United-States-Machine-op-inspct</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  education relationship workclass         occupation native-country gender  \\\n",
       "0      11th    Own-child   Private  Machine-op-inspct  United-States   Male   \n",
       "\n",
       "     education_occupation        native-country_occupation  \n",
       "0  11th-Machine-op-inspct  United-States-Machine-op-inspct  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "wide_preprocessor.inverse_transform(X_wide[:1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, `wide_preprocessor` numerically encodes the `wide_cols` and the `crossed_cols`, which can be recovered using the method `inverse_transform`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  2. TabPreprocessor\n",
    "\n",
    "The `TabPreprocessor` has a lot of different functionalities. Let's explore some of them in detail. In its basic use, the `TabPreprocessor` simply label encodes the categorical columns and normalises the numerical ones (unless otherwised specified)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pytorch_widedeep.preprocessing import TabPreprocessor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# cat_embed_cols = [(column_name, embed_dim), ...]\n",
    "cat_embed_cols = [\n",
    "    (\"education\", 10),\n",
    "    (\"relationship\", 8),\n",
    "    (\"workclass\", 10),\n",
    "    (\"occupation\", 10),\n",
    "    (\"native-country\", 10),\n",
    "]\n",
    "continuous_cols = [\"age\", \"hours-per-week\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "tab_preprocessor = TabPreprocessor(\n",
    "    cat_embed_cols=cat_embed_cols,\n",
    "    continuous_cols=continuous_cols,\n",
    "    cols_to_scale=[\"age\"],  # or scale=True or cols_to_scale=continuous_cols\n",
    ")\n",
    "X_tab = tab_preprocessor.fit_transform(df)\n",
    "# From here on, any new observation can be prepared by simply running `.transform`\n",
    "# new_X_deep = deep_preprocessor.transform(new_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 1.00000000e+00,  1.00000000e+00,  1.00000000e+00, ...,\n",
       "         1.00000000e+00, -9.95128932e-01,  4.00000000e+01],\n",
       "       [ 2.00000000e+00,  2.00000000e+00,  1.00000000e+00, ...,\n",
       "         1.00000000e+00, -4.69415091e-02,  5.00000000e+01],\n",
       "       [ 3.00000000e+00,  2.00000000e+00,  2.00000000e+00, ...,\n",
       "         1.00000000e+00, -7.76316450e-01,  4.00000000e+01],\n",
       "       ...,\n",
       "       [ 2.00000000e+00,  4.00000000e+00,  1.00000000e+00, ...,\n",
       "         1.00000000e+00,  1.41180837e+00,  4.00000000e+01],\n",
       "       [ 2.00000000e+00,  1.00000000e+00,  1.00000000e+00, ...,\n",
       "         1.00000000e+00, -1.21394141e+00,  2.00000000e+01],\n",
       "       [ 2.00000000e+00,  5.00000000e+00,  7.00000000e+00, ...,\n",
       "         1.00000000e+00,  9.74183408e-01,  4.00000000e+01]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_tab"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the label encoding starts from `1`. This is because it is convenient to leave `0` for padding, i.e. unknown categories. Let's take from example the first entry"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 1.        ,  1.        ,  1.        ,  1.        ,  1.        ,\n",
       "       -0.99512893, 40.        ])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_tab[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>education</th>\n",
       "      <th>relationship</th>\n",
       "      <th>workclass</th>\n",
       "      <th>occupation</th>\n",
       "      <th>native-country</th>\n",
       "      <th>age</th>\n",
       "      <th>hours-per-week</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>11th</td>\n",
       "      <td>Own-child</td>\n",
       "      <td>Private</td>\n",
       "      <td>Machine-op-inspct</td>\n",
       "      <td>United-States</td>\n",
       "      <td>25.0</td>\n",
       "      <td>40.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  education relationship workclass         occupation native-country   age  \\\n",
       "0      11th    Own-child   Private  Machine-op-inspct  United-States  25.0   \n",
       "\n",
       "   hours-per-week  \n",
       "0            40.0  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tab_preprocessor.inverse_transform(X_tab[:1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `TabPreprocessor` will have a series of useful attributes that can later be used when instantiating the different Tabular Models, such us for example, the column indexes (used to slice the tensors, internally in the models) or the categorical embeddings set up"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'education': 0,\n",
       " 'relationship': 1,\n",
       " 'workclass': 2,\n",
       " 'occupation': 3,\n",
       " 'native-country': 4,\n",
       " 'age': 5,\n",
       " 'hours-per-week': 6}"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tab_preprocessor.column_idx"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('education', 16, 10),\n",
       " ('relationship', 6, 8),\n",
       " ('workclass', 9, 10),\n",
       " ('occupation', 15, 10),\n",
       " ('native-country', 42, 10)]"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# column name, num unique, embedding dim\n",
    "tab_preprocessor.cat_embed_input"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As I mentioned, there is more one can do, such as for example, quantize (or bucketize) the continuous cols. For this we could use the `quantization_setup` param. This parameter accepts a number of different inputs and uses `pd.cut` under the hood to quantize the continuous cols. For more info, please, read the docs. Let's use it here to quantize \"age\" and \"hours-per-week\" in 4 and 5 \"buckets\" respectively"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/javierrodriguezzaurin/Projects/pytorch-widedeep/pytorch_widedeep/preprocessing/tab_preprocessor.py:358: UserWarning: Continuous columns will not be normalised\n",
      "  warnings.warn(\"Continuous columns will not be normalised\")\n"
     ]
    }
   ],
   "source": [
    "quantization_setup = {\n",
    "    \"age\": 4,\n",
    "    \"hours-per-week\": 5,\n",
    "}  # you can also pass a list of floats with the boundaries if you wanted\n",
    "quant_tab_preprocessor = TabPreprocessor(\n",
    "    cat_embed_cols=cat_embed_cols,\n",
    "    continuous_cols=continuous_cols,\n",
    "    quantization_setup=quantization_setup,\n",
    ")\n",
    "qX_tab = quant_tab_preprocessor.fit_transform(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1, 1, 1, ..., 1, 1, 2],\n",
       "       [2, 2, 1, ..., 1, 2, 3],\n",
       "       [3, 2, 2, ..., 1, 1, 2],\n",
       "       ...,\n",
       "       [2, 4, 1, ..., 1, 3, 2],\n",
       "       [2, 1, 1, ..., 1, 1, 1],\n",
       "       [2, 5, 7, ..., 1, 2, 2]])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "qX_tab"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the continuous columns that have been bucketised into quantiles are treated as any other categorical column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('education', 16, 10),\n",
       " ('relationship', 6, 8),\n",
       " ('workclass', 9, 10),\n",
       " ('occupation', 15, 10),\n",
       " ('native-country', 42, 10),\n",
       " ('age', 4, 4),\n",
       " ('hours-per-week', 5, 4)]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "quant_tab_preprocessor.cat_embed_input"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Where the column 'age' has now 4 categories, which will be encoded using embeddings of 4 dims. Note that, as any other categorical columns, the categorical \"counter\" starts with 1. This is because all incoming values that are lower/higher than the existing lowest/highest value in the train (or already seen) dataset, will be encoded as 0. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 2, 3, 4])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.unique(qX_tab[:, quant_tab_preprocessor.column_idx[\"age\"]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, if we now wanted to `inverse_transform` the transformed array into the original dataframe, we could still do it, but the continuous, bucketized columns will be transformed back to the middle of their quantile/bucket range"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note that quantized cols will be turned into the mid point of the corresponding bin\n"
     ]
    }
   ],
   "source": [
    "df_decoded = quant_tab_preprocessor.inverse_transform(qX_tab)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>fnlwgt</th>\n",
       "      <th>education</th>\n",
       "      <th>educational-num</th>\n",
       "      <th>marital-status</th>\n",
       "      <th>occupation</th>\n",
       "      <th>relationship</th>\n",
       "      <th>race</th>\n",
       "      <th>gender</th>\n",
       "      <th>capital-gain</th>\n",
       "      <th>capital-loss</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>native-country</th>\n",
       "      <th>income</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>25</td>\n",
       "      <td>Private</td>\n",
       "      <td>226802</td>\n",
       "      <td>11th</td>\n",
       "      <td>7</td>\n",
       "      <td>Never-married</td>\n",
       "      <td>Machine-op-inspct</td>\n",
       "      <td>Own-child</td>\n",
       "      <td>Black</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>38</td>\n",
       "      <td>Private</td>\n",
       "      <td>89814</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>9</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Farming-fishing</td>\n",
       "      <td>Husband</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>50</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age workclass  fnlwgt education  educational-num      marital-status  \\\n",
       "0   25   Private  226802      11th                7       Never-married   \n",
       "1   38   Private   89814   HS-grad                9  Married-civ-spouse   \n",
       "\n",
       "          occupation relationship   race gender  capital-gain  capital-loss  \\\n",
       "0  Machine-op-inspct    Own-child  Black   Male             0             0   \n",
       "1    Farming-fishing      Husband  White   Male             0             0   \n",
       "\n",
       "   hours-per-week native-country income  \n",
       "0              40  United-States  <=50K  \n",
       "1              50  United-States  <=50K  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>education</th>\n",
       "      <th>relationship</th>\n",
       "      <th>workclass</th>\n",
       "      <th>occupation</th>\n",
       "      <th>native-country</th>\n",
       "      <th>age</th>\n",
       "      <th>hours-per-week</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>11th</td>\n",
       "      <td>Own-child</td>\n",
       "      <td>Private</td>\n",
       "      <td>Machine-op-inspct</td>\n",
       "      <td>United-States</td>\n",
       "      <td>26.0885</td>\n",
       "      <td>30.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>HS-grad</td>\n",
       "      <td>Husband</td>\n",
       "      <td>Private</td>\n",
       "      <td>Farming-fishing</td>\n",
       "      <td>United-States</td>\n",
       "      <td>44.3750</td>\n",
       "      <td>50.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  education relationship workclass         occupation native-country      age  \\\n",
       "0      11th    Own-child   Private  Machine-op-inspct  United-States  26.0885   \n",
       "1   HS-grad      Husband   Private    Farming-fishing  United-States  44.3750   \n",
       "\n",
       "   hours-per-week  \n",
       "0            30.4  \n",
       "1            50.0  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_decoded.head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "there is one final comment to make regarding to the `inverse_transform` functionality. As we mentioned before, the encoding `0` is reserved for values that fall outside the range covered by the data we used to run the `fit` method. For example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(17, 90)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.age.min(), df.age.max()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "All future age values outside that range will be encoded as 0 and decoded as `NaN`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "tmp_df = df.head(1).copy()\n",
    "tmp_df.loc[:, \"age\"] = 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>fnlwgt</th>\n",
       "      <th>education</th>\n",
       "      <th>educational-num</th>\n",
       "      <th>marital-status</th>\n",
       "      <th>occupation</th>\n",
       "      <th>relationship</th>\n",
       "      <th>race</th>\n",
       "      <th>gender</th>\n",
       "      <th>capital-gain</th>\n",
       "      <th>capital-loss</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>native-country</th>\n",
       "      <th>income</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5</td>\n",
       "      <td>Private</td>\n",
       "      <td>226802</td>\n",
       "      <td>11th</td>\n",
       "      <td>7</td>\n",
       "      <td>Never-married</td>\n",
       "      <td>Machine-op-inspct</td>\n",
       "      <td>Own-child</td>\n",
       "      <td>Black</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age workclass  fnlwgt education  educational-num marital-status  \\\n",
       "0    5   Private  226802      11th                7  Never-married   \n",
       "\n",
       "          occupation relationship   race gender  capital-gain  capital-loss  \\\n",
       "0  Machine-op-inspct    Own-child  Black   Male             0             0   \n",
       "\n",
       "   hours-per-week native-country income  \n",
       "0              40  United-States  <=50K  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# quant_tab_preprocessor has already been fitted with a data that has an age range between 17 and 90\n",
    "tmp_qX_tab = quant_tab_preprocessor.transform(tmp_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1, 1, 1, 1, 1, 0, 2]])"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tmp_qX_tab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note that quantized cols will be turned into the mid point of the corresponding bin\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>education</th>\n",
       "      <th>relationship</th>\n",
       "      <th>workclass</th>\n",
       "      <th>occupation</th>\n",
       "      <th>native-country</th>\n",
       "      <th>age</th>\n",
       "      <th>hours-per-week</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>11th</td>\n",
       "      <td>Own-child</td>\n",
       "      <td>Private</td>\n",
       "      <td>Machine-op-inspct</td>\n",
       "      <td>United-States</td>\n",
       "      <td>NaN</td>\n",
       "      <td>30.4</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  education relationship workclass         occupation native-country  age  \\\n",
       "0      11th    Own-child   Private  Machine-op-inspct  United-States  NaN   \n",
       "\n",
       "   hours-per-week  \n",
       "0            30.4  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "quant_tab_preprocessor.inverse_transform(tmp_qX_tab)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##  3. TextPreprocessor\n",
    "\n",
    "This preprocessor returns the tokenised, padded sequences that will be directly fed to the stack of LSTMs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pytorch_widedeep.preprocessing import TextPreprocessor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The airbnb dataset, which you could get from here:\n",
    "# http://insideairbnb.com/get-the-data.html, is too big to be included in\n",
    "# our datasets module (when including images). Therefore, go there,\n",
    "# download it, and use the download_images.py script to get the images\n",
    "# and the airbnb_data_processing.py to process the data. We'll find\n",
    "# better datasets in the future ;). Note that here we are only using a\n",
    "# small sample to illustrate the use, so PLEASE ignore the results, just\n",
    "# focus on usage\n",
    "df = pd.read_csv(\"../tmp_data/airbnb/airbnb_sample.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[\"My bright double bedroom with a large window has a relaxed feeling! It comfortably fits one or two and is centrally located just two blocks from Finsbury Park. Enjoy great restaurants in the area and easy access to easy transport tubes, trains and buses. Babies and children of all ages are welcome. Hello Everyone, I'm offering my lovely double bedroom in Finsbury Park area (zone 2) for let in a shared apartment.  You will share the apartment with me and it is fully furnished with a self catering kitchen. Two people can easily sleep well as the room has a queen size bed. I also have a travel cot for a baby for guest with small children.  I will require a deposit up front as a security gesture on both our parts and will be given back to you when you return the keys.  I trust anyone who will be responding to this add would treat my home with care and respect .  Best Wishes  Alina Guest will have access to the self catering kitchen and bathroom. There is the flat is equipped wifi internet,\",\n",
       " \"Lots of windows and light.  St Luke's Gardens are at the end of the block, and the river not too far the other way. Ten minutes walk if you go slowly. Buses to everywhere round the corner and shops, restaurants, pubs, the cinema and Waitrose . Bright Chelsea Apartment  This is a bright one bedroom ground floor apartment in an interesting listed building. There is one double bedroom and a living room/kitchen The apartment has a full  bathroom and the kitchen is fully equipped. Two wardrobes are available exclusively for guests and bedside tables and two long drawers. This sunny convenient compact flat is just around the corner from the Waitrose supermarket and all sorts of shops, cinemas, restaurants and pubs.  This is a lovely part of London. There is a fun farmers market in the King's Road at the weekend.  Buses to everywhere are just round the corner, and two underground stations are within ten minutes walk. There is a very nice pub round by St. Luke's gardens, 4 mins slow walk, the \"]"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "texts = df.description.tolist()\n",
    "texts[:2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The vocabulary contains 2192 tokens\n"
     ]
    }
   ],
   "source": [
    "text_preprocessor = TextPreprocessor(text_col=\"description\")\n",
    "X_text = text_preprocessor.fit_transform(df)\n",
    "# From here on, any new observation can be prepared by simply running `.transform`\n",
    "# new_X_text = text_preprocessor.transform(new_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[  29   48   37  367  818   17  910   17  177   15  122  349   53  879\n",
      " 1174  126  393   40  911    0   23  228   71  819    9   53   55 1380\n",
      "  225   11   18  308   18 1564   10  755    0  942  239   53   55    0\n",
      "   11   36 1013  277 1974   70   62   15 1475    9  943    5  251    5\n",
      "    0    5    0    5  177   53   37   75   11   10  294  726   32    9\n",
      "   42    5   25   12   10   22   12  136  100  145]\n"
     ]
    }
   ],
   "source": [
    "print(X_text[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. ImagePreprocessor\n",
    "\n",
    "`ImagePreprocessor` simply resizes the images, being aware of the aspect ratio.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pytorch_widedeep.preprocessing import ImagePreprocessor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Reading Images from ../tmp_data/airbnb/property_picture/\n",
      "Resizing\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100%|█████████████████████████████████████████████████████████████████████████████████████████████████████| 1001/1001 [00:01<00:00, 667.89it/s]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Computing normalisation metrics\n"
     ]
    }
   ],
   "source": [
    "image_preprocessor = wd.preprocessing.ImagePreprocessor(\n",
    "    img_col=\"id\", img_path=\"../tmp_data/airbnb/property_picture/\"\n",
    ")\n",
    "X_images = image_preprocessor.fit_transform(df)\n",
    "# From here on, any new observation can be prepared by simply running `.transform`\n",
    "# new_X_images = image_preprocessor.transform(new_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(224, 224, 3)"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_images[0].shape"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "3b99005fd577fa40f3cce433b2b92303885900e634b2b5344c07c59d06c8792d"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
